Feed a Transformer's own top-layer states back in as input, and every FLOP and every token count for more: the same loss with up to 46% less compute or 20% less unique data.
